The squint angle, also known as yaw angle, between the left camera and the right camera in a stereo camera system, must be determined with great accuracy, because an error in this angle results in large distance estimation errors in the stereo calculations. The distance error grows with the square of the distance. For example, for a 160 mm baseline stereo camera, a squint angle error as low as 0.01° gives a distance estimation error of around 10 m at a distance of 100 m. For an automotive stereo camera the squint angle will not be constant over the vehicle life time due to thermal changes and the long life time of automotive systems. Therefore, an online solution for estimating a squint angle error is needed.
It is known to estimate the squint angle error using radar or lidar distance information as a reference value, which however requires a radar or lidar reference system.
DE 10 2012 009 577 A1 describes a method of calibrating the squint angle between two stereo cameras in a motor vehicle from a comparison of a reference driven distance determined from odometric data to a stereoscopic driven distance determined by image processing. However, the external odometric data needed to calculate the reference driven distance constitute an additional systematic source of uncertainty for the determination of the squint angle error.
US 2014/0168377 A1 discloses a method for aligning a stereo camera of a vehicle mounted object detection system, wherein an image from each camera at two different times is used to determine an observed displacement of a stationary object, like a traffic sign, relative to the vehicle. A predicted displacement of the object relative to the vehicle is also determined using, for example, a difference of size of the object in images taken at two different times. Determining a triangulation correction based on a difference of the observed displacement and the predicted displacement is used to correct for misalignment of the cameras.
An object of the present invention is to provide a vision system and a method of controlling a vision system which allow for an accurate determination of a squint angle error between the stereo cameras during operation of a motor vehicle.
An embodiment of the invention solves this object with the features described in the specification and shown by the appended drawings. According to an embodiment, a regression analysis using a predetermined regression function having at least one variable is performed on a group of at least five data points of the tracked object corresponding to different times. Herein, each data point establishes a size related value and a corresponding, i.e. same-time, disparity related value of the detected object. The regression analysis yields a best value for each variable corresponding to a best match or fit of the regression function to the group of data points. A systematic error in the disparity related values and/or in the size related values is calculated from the at least one best value. This systematic error can provide a measure for the squint angle error, derived from the image data taken by the imaging system, only, without any reference to external information like external odometric data, ego vehicle speed, or radar or lidar information.
An embodiment of the invention uses size information from objects in the image that are tracked over time. The size information of an object together with the calculated stereo information (disparity or disparity related information) of the corresponding object is used to estimate the squint angle error, or a systematic error unambiguously related to it.
If the value of the squint angle error is explicitly required, the determined systematic error in the disparity related values and/or the size related values into a squint angle error can easily be converted to the squint angle error using the known optical parameters of the imaging system.
In comparison to the prior art described for example in US 2014/0168377 A1, the invention yields a much more accurate measure of the squint or yaw angle error since it is based on not only two data points, but on at least five data points, preferably at least ten data points.
Furthermore, the invention is not limited to the investigation of stationary objects. Preferably the tracked object used in the regression analysis is another vehicle which may be a moving vehicle, in particular an oncoming vehicle. Other objects than vehicles may be used in the regression analysis.
Preferably the regression analysis is a linear regression analysis. In a preferred embodiment, the regression function is a straight line. Here, the following facts are advantageously used; 1) For a perfect squint angle (zero squint angle error) the width of an object and the disparity are both zero at infinite distance; and 2) For a perfect squint angle the width doubles when the distance is halved. In particular, if the disparity related values are disparity values and the size related values are size values, the ideal relationship is a straight line. In this case, the systematic error in the disparity related values and/or the size related values can easily be determined as an offset of the regression function to the origin, i.e. zero disparity and zero size.
Preferably the regression analysis is performed step by step, i.e. finding the best value of a first variable of the regression function, then finding the best value of a second variable of the regression function, and so on. This may be more effective than performing the regression analysis for all variables simultaneously in a multi-variable regression, which may be too computationally demanding given the limited processing capabilities in a motor vehicle.
In particular, if the regression function is a straight line, the regression analysis preferably includes a first regression step of finding the best slope of the straight line regression function, and a second regression step of finding the best offset of the straight line regression function while keeping the slope fixed to the best slope established in the first regression step.
Since the slope of the straight line is determined by the absolute size of the tracked object, the step of finding the best slope of the straight line regression function can advantageously be dispensed with if the absolute size of the tracked object is known from other sources, for example through inter-vehicle communication, or from image processing. In that case, the regression analysis preferably reduces to a cost-saving single variable regression analysis of finding the best offset of the straight line regression function.
The data points used in the regression analysis are preferably spanned over a range corresponding to a distance range of at least 30 m, more preferably at least 50 m, even more preferably at least 70 m. Generally, the precision in the determination of the systematic error can be higher if the distance range of the data points taken into account in the regression analysis is larger. Nevertheless, preferably data points corresponding to an object distance of less than a predetermined value, which advantageously may be in the range of 5 m to 35 m, preferably 10 m to 30 m, more preferably 15 m to 25 m, for example 20 m, can be discarded in the regression analysis, since data points corresponding to a relatively close distance to the tracked object show increasing deviations of other origin predominating over the deviation due to a squint angle error.
In a preferred embodiment, the disparity related values are disparity values. In another embodiment, the disparity related values can be for example distance values. Preferably the size related values are object sizes, in particular object widths. In another embodiment, the size related values can be for example object heights.
Preferably, the regression results from several tracked vehicles are combined to yield a more stable and accurate systematic error value.